Skip to content

feat(observations)!: dated events never collapse across dates (D106) - #360

Merged
fazpu merged 3 commits into
mainfrom
fix/recurring-event-adjudication
Sep 3, 2026
Merged

feat(observations)!: dated events never collapse across dates (D106)#360
fazpu merged 3 commits into
mainfrom
fix/recurring-event-adjudication

Conversation

@fazpu

@fazpu fazpu commented Sep 3, 2026

Copy link
Copy Markdown
Member

Problem

The observation adjudicator (D43) decides whether a new entity-anchored statement re-asserts an existing one by sending the small model exactly two strings — EXISTING and NEW — with no dates, even though every claim carries a resolved D41 valid-time window (claim_valid_kind/claim_valid_from/claim_valid_until).

On the v0.11.0 LoCoMo conv-42 run (plan/analysis/locomo_conv42_recurring_event_adjudication.md), extraction captured all seven of Nate's tournament wins as clean, dated claims and identity resolved one Nate — yet the fact layer kept four win facts. The observation_adjudications transcript shows the small model judged October's "won a really big video game tournament last week" to be evidence for January's "won his first video game tournament last week" (both say "last week"), and folded the international and Valorant wins into "has been winning a few gaming tournaments". Ten participation lineages collapsed into the header boilerplate "Nate is a participant.", swallowing two real tournament entries. The counting question was answered "At least five" against seven.

Change (decision D106)

A deterministic temporal-compatibility rung in spine/observation_adjudication.py that bounds what a verdict may do, using the D41 windows the claims already carry — the "deterministic value/period compare" step the D43 cascade always named but never implemented:

  • two dated events with disjoint resolved windows are different occurrences unless the date-aware model finds they name the same occurrence with disputed dates: the pair may only contradict (both stand, grouped) or stay new; evidence/supersede are coerced to new and recorded. Byte-identical text is then kept apart without any model call;
  • a dated event vs an undated statement may still supersede or contradict (a dated resignation can end a "is CEO" state), but evidence is coerced to new — a specific dated event never re-asserts a vaguer summary, and a summary never re-asserts a specific event; identical text is kept apart without a model call;
  • two undated statements, or overlapping dated windows (including year-level + day-level for one occurrence), behave exactly as before;
  • a D41 open interval (start, no end) stays unbounded, never a point.

The verdict prompt shows, for each statement, when the source said it and what world-time it is about (the resolved window of any D41 kind), and defines both clocks. _BLOCK_ENTITY derives each observation's windows from its supporting current-testimony claims; in-batch candidates carry timing on every insert and widen as they absorb evidence; every verdict record carries the coercions before it.

Versions roll (provenance, D12): OBSERVATION_ADJUDICATOR_VERSIONobs-adjudicator-2026.09a:temp0-1:temporal-gate-1; OBS_FLUSH_VERSIONe3-obs-flush-2026.09a:temporal-gate-1:claim-fanout-1:entity-fanout-1; LoCoMo protocol Full-v20 → Full-v21 (precedent: D100/D102/D104/D105). Rollout is stop-drain-rebuild: the entity flush handler now reports the claimed unit's own generation to its barrier, so units enqueued before the roll drain cleanly; readiness reports the old generation until rebuild.

What is deliberately not changed

  • Observation valid_from stays the claim's asserted_at; using the resolved window as the observation's own validity is a separate decision.
  • E2 selection of document-framing claims ("Participants: Joanna and Nate") — source-faithful testimony is kept (D32); the defect was absorption, not extraction, and the rung makes the boilerplate harmless.
  • The reader's reasoning effort / the counting question itself — this PR completes the fact layer; whether the reader then counts correctly is the next run's question.

Tests

Thirteen test_d106_* proofs in src/tests/spine/test_observation_adjudication.py, mostly driven by a collapse-happy fake model that answers evidence for any tournament pair, so they pass only if the rung makes the model's answer irrelevant: seven dated wins → seven facts; arrival-order invariance; same-event re-mention still collapses; identical words about two dates → two events (no model); identical text dated-beside-undated coexists (no model); vague summary never absorbs a dated event (and the mirror), asserting the prompt's said on/is about lines and the recorded coercion; boilerplate never absorbs dated participation; open window stays unbounded; year+day overlap may collapse; in-batch widening (one row, three evidence links, zero verdicts); disjoint dates may still contradict one occurrence (shared group); a dated event may supersede an undated state. Plus a rollout proof that the flush handler reports the claimed unit's generation. All existing D43/D88/D90 proofs unchanged and green.

Validation: ruff check, ruff format --check, pyright clean repo-wide. Targeted suites (adjudicator, E3, protocol, runner) green. Full suite against the CI PostgreSQL image: 2,185 passed; test_client_sdk::test_sdk_pushes_lineage_metadata_to_e0 also fails on unmodified main in this environment (MIME detection); six test_query_space_batch_a proofs failed in the full run but pass alone and in sequence with the adjudication module on this branch — a full rerun with captured tracebacks is in progress and will be reported here before merge.

Contributor agreement

Review

Codex (gpt-5.6-sol, xhigh) review and triage are posted as comments; all five majors are addressed by behavior changes with tests. Antigravity could not run (account quota exhausted, resets in ~4 days).

Corpus (same PR)

decisions.md D106 (+ D78 banner) · plan/designs/observations_design.md §3 rung + worked example · plan/designs/locomo_benchmark_design.md D106 amendment (D105 historical) · plan/analysis/locomo_conv42_recurring_event_adjudication.md · plan/plans/phase-2-truth-machinery.md WP-2.5 note · benchmarks/locomo/README.md V21 · website/.../project-status bullet.

🤖 Generated with Claude Code

https://claude.ai/code/session_016p9xB9T7VYREuVmohDPKjR

fazpu and others added 2 commits September 3, 2026 11:05
The observation adjudicator judged whether a new statement re-asserts an
existing one from two bare strings, never reading the D41 event window
every claim already carries. On LoCoMo conv-42 that folded three of seven
dated tournament wins into earlier facts — October's "won a really big
tournament last week" became evidence for January's "won his first
tournament last week" because both say "last week" — and ten participation
lineages collapsed into header boilerplate, swallowing two real entries.

Add the deterministic temporal-compatibility rung D43's design named but
never implemented: two dated events with disjoint resolved windows never
interact and buy no verdict (also overriding the exact-statement shortcut);
a dated event paired with an undated statement may still be judged for
supersede/contradict, but an `evidence` verdict is coerced to `new` and
recorded; undated pairs and overlapping-window events behave as before.
The verdict prompt now shows both timelines. The candidate block derives
each observation's event window from its supporting event-time claims.

Roll the adjudicator generation, the adjudicate_observations flush
component version, and the LoCoMo protocol (Full-v20 -> Full-v21). Add
seven D106 fixture tests driven by a collapse-happy fake model, the D106
decision, the observations design amendment and worked example, the
conv-42 evidence analysis, and the docs/status updates.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016p9xB9T7VYREuVmohDPKjR
…pping pairs (D106 review)

Codex review of #360 found the first cut over-broad: disjoint dated
events were declared unrelated without a verdict, which would double-count
one occurrence whose date two sources dispute and suppress its
contradiction group. Disjoint pairs now reach the date-aware model and may
only contradict or stay new; evidence and supersede are coerced to new and
recorded. The exact-statement shortcut requires temporal compatibility
(identical text beside a dated/undated or disjoint-window twin coexists,
decided without a model). Open-ended D41 windows stay unbounded instead of
collapsing to a point. In-batch candidates carry timing on every insert and
widen their windows as they absorb evidence. Every verdict record carries
the coercions before it.

The entity flush handler reports the claimed unit's own component
generation to its barrier so units enqueued before the OBS_FLUSH_VERSION
roll drain under the generation their barrier counts; D106 states the
stop-drain-rebuild rollout. The verdict prompt labels each side's two
clocks ("said on" — the source's date; "is about" — the resolved world-time
of any D41 kind) and defines them. Six more proofs; stale v20-as-current
references corrected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016p9xB9T7VYREuVmohDPKjR
@fazpu

fazpu commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Independent review — Codex (gpt-5.6-sol, reasoning xhigh, review-only run against commit 02c9696)

Antigravity could not run: its CLI returned Individual quota reached … Resets in 98h (empty output). The second independent review is therefore missing for quota reasons, not skipped.

Scope: origin/main...HEAD; line numbers refer to committed HEAD. A prompt-only uncommitted edit appeared during review and was excluded.

  1. major — decisions.md D106, lines 4899–4914; plan/designs/observations_design.md §3, lines 146–163; src/rememberstack/spine/observation_adjudication.py, lines 473–494. Disjoint event windows do not prove that two statements describe different event instances. Two sources can disagree about the date of one uniquely identified event—e.g. “Nate won the 2025 Valorant final on Friday” versus “…on Saturday”—or report a rescheduled named meeting. D106 states that disjoint events “never interact,” so the implementation skips evidence, supersession, and contradiction before inspecting event identity. That can double-count one event and suppress a required contradiction group. D43’s no-cap rule justifies refusing an unsafe overwrite; it does not justify declaring the claims unrelated. Suggested fix: revise D106 so disjointness forbids evidence-collapse by itself but does not rule out same-instance contradiction/correction. Without a deterministic event-instance identifier, semantically similar disjoint pairs must reach the date-aware adjudicator; alternatively add a reliable instance key before retaining the zero-call shortcut.

  2. major — src/rememberstack/spine/observation_adjudication.py, lines 314–344. The exact-statement shortcut still collapses mixed-dating pairs, directly violating D106. Its only temporal condition is not _disjoint_dated_events(...); that function returns false when exactly one side is dated, so byte-identical dated/undated testimony immediately receives _evidence() without reaching the mixed-dating coercion at lines 495–517. For example, one anchored and one unanchored extraction of “Alice resigned yesterday” become one observation, although D106 requires the unanchored re-mention to coexist. The new mixed tests use different strings and therefore miss this. Suggested fix: make exact evidence conditional on temporal compatibility: both undated, or both dated with overlapping windows. Route mixed exact matches through the ladder/coercion path, or deterministically insert a duplicate if supersede/contradict is not meaningful.

  3. major — src/rememberstack/spine/observation_adjudication.py, lines 1144–1168 and 1288–1300; D41 schema contract in plan/designs/postgres_schema_design.md §8, lines 1510–1530. Open-ended D41 event windows are collapsed to a point. _timing_from_row() and _candidate_timing() replace claim_valid_until IS NULL with event_from, while _BLOCK_ENTITY uses max(coalesce(claim_valid_until, claim_valid_from)). D41 explicitly defines precision open as a non-null start with a null, unbounded upper endpoint. Consequently [2026-01-01, ∞) is treated as [2026-01-01, 2026-01-01] and incorrectly considered disjoint from a February event. Suggested fix: retain None as positive infinity; make _windows_disjoint() compare an upper endpoint only when it is non-null; render it as “from … onward”; and aggregate event_until as null whenever any supporting current event claim is open, otherwise use max(claim_valid_until). Add single-claim and mixed bounded/open aggregate tests.

  4. major — src/rememberstack/spine/observation_adjudication.py, lines 518–545, 698–703, 746–752, and 1096–1117; src/tests/spine/test_observation_adjudication.py, lines 771–1001. The front-loaded in-transaction candidate list does not remain temporally equivalent to the database block. Forward supersede and contradiction call _remember_candidate() without timing, making newly inserted dated observations appear undated to later assertions. Exact/model evidence-collapse also adds a supporting event claim without widening the candidate’s in-memory event_from/event_until. A reproducible chain is [Jan 1–10], an identical overlapping re-mention [Jan 10–20], then [Jan 20]: the persisted aggregate spans through Jan 20, but the stale candidate still ends Jan 10 and the third claim is incorrectly split. All seven D106 tests call _add()/add_observation() separately, reloading the database block each time, so none exercises this production batch behavior or the D90 re-split list. Suggested fix: centralize candidate insertion/update, always pass timing, and merge newly accepted evidence windows into the candidate. Add one-call add_observations() and flush_entity_global_staging() tests covering overlapping chains, supersede, contradiction, and dated D90 re-splitting.

  5. major — src/rememberstack/workers/e3.py, lines 597–606 and 651–665; src/rememberstack/spine/work_ledger.py, lines 1391–1403, 447–464, and 1814–1828; decisions.md D106 Consequences, lines 4941–4949. The flush-version roll is unsafe when old entity-unit work remains pending. claim_one() selects any component generation; the handler accepts every entity-targeted observation job, but reports the new constant in EntityObsFlushBarrier. Completion marks the actually claimed old-version row succeeded and then counts succeeded rows at the new version, so the barrier never completes and downstream supersession/embed work can remain unopened. Fully completed old stores also report the new expected component as missing until rebuilt, while existing membership/state is keyed by normalizer version and prevents ordinary re-fan-out. The decision only says stores re-adjudicate “on rebuild” and omits this drain/readiness/staging contract. Suggested fix: define and test a stop-drain-rebuild rollout. The handler should fail closed before consuming staging when work.component_version != OBS_FLUSH_VERSION, and the rebuild procedure must explicitly reconstruct or clear staging, membership, version state, and readiness rows before enqueuing the new generation.

  6. minor — src/rememberstack/spine/observation_adjudication.py, lines 483–517 and 755–769. The required temporal-coercion audit is conditional and sometimes mislabeled. temporal_skips is persisted only when no candidate interacts; if a mixed pair’s evidence is coerced and a later candidate supersedes, contradicts, or absorbs the claim, the coercion disappears from the adjudication transcript. When every candidate is deterministically skipped with zero verdict calls, the inserted row is nevertheless recorded with method="small_model". Suggested fix: attach accumulated temporal decisions to every terminal record and use a truthful temporal_gate method when no model ran. Add a multi-candidate test where coercion precedes a later interaction.

  7. minor — src/rememberstack/spine/observation_adjudication.py, prompt lines 75–82; decisions.md lines 4912–4914. The prompt says events with “DIFFERENT resolved event dates” are always new and that only the “same resolved date” can be evidence. The binding rule is narrower: only disjoint windows are deterministic; overlapping windows—including a coarse year/month claim and a more precise day claim—must be adjudicated as before. This wording biases the model toward duplicates on compatible overlapping windows. Suggested fix: say “disjoint/non-overlapping resolved event windows” and explicitly allow differently precise overlapping windows to be evidence when the statements identify the same occurrence.

  8. minor — src/tests/spine/test_observation_adjudication.py, lines 771–1001. Important contract branches remain uncovered: mixed dating with identical text, open-ended windows, overlapping multi-day windows, a mixed pair returning supersede or contradict, dated evidence through D90 re-splitting, and two same-day statements that must remain distinct or contradict. The current same-event test exercises only the exact shortcut. Suggested fix: add focused tests for those cases, using batch APIs where applicable and asserting caps, contradiction groups, evidence placement, audit features, and model-call counts.

  9. nit — decisions.md lines 2913–2915; plan/designs/locomo_benchmark_design.md line 67; src/tests/benchmarks/test_locomo_protocol.py line 242. A few v20 references still present v20 as current: D78’s banner says “The current protocol is … v20,” the design says accepting it does not authorize a paid “v20” run, and the v21 test remains named test_protocol_is_v20.... Suggested fix: add the D106 current amendment above D78’s historical D105 note, change the execution warning to v21 or protocol-neutral wording, and rename the test.

Verified

  • Bounded claim windows use correct inclusive closed-interval disjointness: touching endpoints overlap, and PostgreSQL supplies comparable timestamptz values.
  • _BLOCK_ENTITY derives bounded spans only from supporting, current-testimony event_time claims; undated support is ignored, withdrawn testimony is excluded, and observations with no current dated support become undated.
  • Null claim_valid_kind, missing claim rows, and unresolved event windows degrade safely to undated rather than raising.
  • D43’s existing cap safeguards remain intact: supersede requires a non-empty rationale and confidence at or above the margin, and every actual cap receives an adjudication reason row.
  • The exact disjoint-date shortcut override is exercised. The fake embeddings for the seven-win corpus all exceed the novelty floor (minimum pair similarity approximately 0.69), so the zero-verdict assertion is due to the temporal rung rather than clear-novelty bypass.
  • The active LoCoMo protocol, adapter key/types, benchmark runner, self-host readiness component, OBS_FLUSH_VERSION, README, sharding defaults, and protocol assertions consistently pin Full-v21/new flush generation.
  • git diff --check origin/main...HEAD passes. Integration tests were not executed because their fixture downgrades/upgrades and truncates the configured PostgreSQL store, which is outside a mutation-free review.

Merge verdict: request changes; this is not mergeable yet. First resolve the over-broad design assumption that disjoint dates always establish distinct event identity, then fix the mixed exact shortcut, open-ended interval semantics, and in-batch candidate maintenance. The flush-generation rollout must also be made safe and documented before shipping; audit/test/doc corrections should follow those behavioral decisions.

@fazpu

fazpu commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Review triage — dispositions applied in the follow-up commit

Every Codex finding was re-verified against the code before acting. The review changed the design materially (not just wording): the first revision treated disjoint dated events as never interacting; the shipped rule now bounds what a verdict may do and keeps the date-aware model in the loop.

# Finding (Codex) Verified Disposition
1 Disjoint windows don't prove different events — two sources can dispute the date of one occurrence; skipping the pair suppresses the contradiction Yes Fixed (design change). Disjoint dated pairs now reach the date-aware model; only contradict/new are honoured, evidence/supersede are coerced to new and recorded. D106, the observations design §3, prompt, README, and status text all restated. New test: test_d106_disjoint_dates_may_still_contradict_one_occurrence (shared contradiction_group).
2 Exact-statement shortcut still collapses dated/undated identical text Yes — _disjoint_dated_events returned False for mixed pairs Fixed. Exact collapse requires _evidence_compatible (both undated, or both dated & overlapping); otherwise a deterministic coexist row (method='exact', reason recorded), no model call. New test: test_d106_identical_text_dated_beside_undated_coexists.
3 Open-ended D41 windows (until IS NULL) collapsed to a point Yes Fixed. None end stays unbounded in _ClaimTiming, _windows_disjoint, and the block aggregate (CASE WHEN bool_or(until IS NULL) THEN NULL ELSE max(until)); rendered as "from X onward". New test: test_d106_open_window_is_unbounded_not_a_point.
4 In-batch candidates: forward-supersede and contradict inserts lacked timing; evidence collapse didn't widen the in-memory window Yes — two _remember_candidate calls had a different indentation than my replace-all pattern Fixed. All inserts pass timing; _absorb_timing widens event/about windows on every collapse (exact and ladder). New batch test through add_observations: test_d106_batch_widens_the_candidate_window_as_it_absorbs (Jan 1–10 / Jan 10–20 / Jan 20 → one row, three evidence links, zero verdicts).
5 Flush-version roll strands pending old-generation units (handler reports the new constant; barrier counts at that version) Yes — _COUNT_OBS_FLUSH_UNITS_SUCCEEDED filters p.component_version = :obs_flush_version Fixed + documented. _handle_entity_unit reports work.component_version (the claimed unit's own generation) to EntityObsFlushBarrier; new-unit fan-out still enqueues at OBS_FLUSH_VERSION. D106 Consequences now states the stop-drain-rebuild contract (old units drain under their generation; readiness reports the old generation until rebuild). Test: test_entity_handler_reports_the_claimed_units_own_generation.
6 Coercion audit only persisted on the terminal new; method='small_model' when no model ran Yes Fixed. Every per-candidate verdict record carries temporal_relation and the coercions so far (temporal_gate); the identical-text decision records method='exact' (an existing enum value — adjudication_method is a DB enum, so no new value was invented).
7 Prompt overclaims "different dates ⇒ always new" Yes Fixed. Prompt now says non-overlapping windows are different occurrences unless they plainly name the same occurrence with a disputed date (→ contradict), and that overlapping windows of different precision may be the same occurrence.
8 Missing tests: mixed exact, open windows, overlapping precisions, mixed supersede, batch, disjoint contradiction Yes Fixed — six new proofs added (13 test_d106_* total): mixed exact, open window, year+day overlap collapses, batch widening, disjoint-dates contradiction, dated event supersedes undated state. Dated evidence through the D90 re-split path remains covered only by the existing re-split proofs (undated); noted as a follow-up.
9 Stale v20-as-current references (D78 banner, design line 67, test name) Yes Fixed — D106 banner added above the historical D105 note in D78; execution warning made protocol-neutral; test renamed test_protocol_is_v21_….

Also folded in since the review ran: the verdict prompt now labels each side's two clocks explicitly (said on: — the source's own date; is about: — the resolved world-time of any D41 kind, not only events) and defines both, per the operator's reading of the earlier EXISTING TIME: lines.

Codex's verdict was "request changes"; all five majors are addressed by behavior changes with tests, not wording. Full suite, ruff, and pyright are green on the follow-up commit.

…ble fragments

The operational-scale battery gates "one block read and one claim-timing
read per entity batch" by matching literal SQL prefixes. D106 aliases the
observations table and selects the D41 valid-time columns, so the old
prefixes no longer occurred although both statements still run exactly
once. Match stable fragments of the new statements instead; the counts and
transaction gates are unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016p9xB9T7VYREuVmohDPKjR
@fazpu

fazpu commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Validation state at merge

  • CI on 7697011f: all twelve checks pass (Unit, Quality, Contract smoke, Integration adapters/surfaces/workers, build, CLA, PR gate); merge state CLEAN.
  • The Integration (adapters) failure on the previous commit was the operational-scale battery matching literal SQL prefixes that D106 changed (the block and claim-timing statements still run exactly once per batch); fingerprints updated in 7697011f.
  • Local full database suite on the CI PostgreSQL image: 2,185 passed. test_client_sdk::test_sdk_pushes_lineage_metadata_to_e0 fails identically on unmodified main in this environment (MIME detection). Six test_query_space_batch_a proofs failed once in a full local run but pass alone and in sequence with the adjudication module on this branch; two attempts at a full reproduction with captured tracebacks were cut short by the local environment. Residual: if nightly CI reproduces them on main, treat it as a D106 test-ordering interaction and follow up on this decision.

@fazpu
fazpu merged commit 02b7990 into main Sep 3, 2026
12 checks passed
@fazpu
fazpu deleted the fix/recurring-event-adjudication branch September 3, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant